Transport haute-performance: problématique et solutions proposées

L'univers du calcul haute-performance a découvert des limitations du protocole TCP, l'empêchant d'utiliser pleinement les liens haut-débit et grande distance prévus pour former l'ossature des grilles de calcul. Cet article présente ces problèmes de performance, puis décrit, analyse et compare les solutions actuellement proposées par la communauté réseau.RES-

Impact of the Correlation between Flow Rates and Durations on the Large-Scale Properties of Aggregate Network Traffic

Since the discovery of long-range dependence in network traffic in 1993, many models have appeared to reproduce this property, based on heavy-tailed distributions of some flow-scale properties of the traffic. However, none of these models consider the correlation existing between flow rates and flow durations. In this work, we extend previously proposed models to include this correlation. Based on a planar Poisson process setting, which describes the flow-scale traffic structure, we analytically compute the auto-covariance function of the aggregate traffic's bandwidth and show that it exhibits long-range dependence with a different Hurst parameter. In uncorrelated case, the model that we propose is consistent with existing models, and predict the same Hurst parameter. We also prove that pseudo long-range dependence with a different index can arise from highly variable flow rates. The pertinence of our model choices is validated on real web traffic traces

An Efficient Network API for in-Kernel Applications in Clusters

International audienceRunning parallel applications on clusters with high-speed local networks requires fast communication between computing nodes but also low latency and high bandwidth file access. However, the application programming interfaces of high-speed local networks were designed for MPI communication and do not always meet the requirements of other applications like distributed file systems. In this paper, we explore several solutions to improve the use of high-speed network for in-kernel applications. Distributed file systems implemented on top of the GM interface of Myrinet are first examined to demonstrate how hard it is to get an efficient interaction between such applications and the network. Then, we propose solutions to simplify and improve this interaction and integrate them into the kernel interface of the new Myrinet. Performance comparisons between MX and GM, and their usage in both a distributed file system and a zero-copy protocol show nice improvements. Moreover, we are able to improve the performance of the flexible kernel API we designed in MX that allows to remove some intermediate copy

Towards a User-Oriented Benchmark for Transport Protocols Comparison in very High Speed Networks

Standard TCP faces some performance limitations in very high speed wide area networks, mainly due to a long end-to-end feedback loop and a conservative behaviour with respect to congestion. Many TCP variants have been proposed to overcome these limitations. However, TCP is a complex protocol with many user-configurable parameters and a range of different implementations. It is then important to define measurement methods so that the transport services and protocols can evolve guided by scientific principles and compared quantitatively. The goal of this report is to present some steps towards a user-oriented benchmark, called ITB, for high speed transport protocols comparison. We first present and analyse some results reported in the literature. From this study we identify classes of representative applications and useful metrics. We then isolate infrastructure parameters and traffic factors which influence the protocol behaviour. This enable us to define scenario capturing and synthesising comprehensive and useful properties. We finally illustrate this proposal by preliminary results obtained on our experimental environment, Grid'5000, we have built and are using for contributing in this benchmark design

Executing distributed applications on virtualized infrastructures specified with the VXDL language and managed by the HIPerNET framework

International audienceWith the convergence of computing and communication, and the expansion of cloud computing, new models and tools are needed to allow users to define, create, and exploit on-demand virtual infras- tructures within wide area distributed environments. Optimally design- ing customized virtual execution-infrastructure and executing them on a physical substrate remains a complex problem. This paper presents the VXDL language, a language for specifying and describing virtual infras- tructures and the HIPerNET framework to manage them. Based on the example of a specific biomedical application and workflow engine, this paper illustrates how VXDL enables to specify different customized vir- tual infrastructures and the HIPerNET framework to execute them on a distributed substrate. The paper presents experiments of the deploy- ment and execution of this application on different virtual infrastructures managed by our HIPerNet system. All the experiments are performed on the Grid'5000 testbed substrate

Document d'aide au déploiement d'IPv6 sur Grid5000

Ce document présente succinctement les motivations pour l'utilisation d'IPv6 dans les grilles. Une analyse des opérations nécessaires pour son déploiement dans la plateforme Grid5000 est ensuite présentée et détaillée

Etude d'implémentations MPI dans une grille de calcul

National audienceThis paper presents an evaluation on the GRID'5000 platform of four MPI implementation: MPICH2, MPICH-Madeleine, OpenMPI et GridMPI. The comparison is based on a simple pingpong and the NAS parallel benchmarks. We show that GridMPI has the best results with regards to performances on the Grid if the implementation is well tuned. This paper details which parameters should be tuned.De nos jours, les grappes de PC ou clusters sont souvent interconnectés par des réseaux longue-distance de manière à former une grille afin d'offrir à un grand nombre d'utilisateurs un nombre plus conséquent de ressources. MPI, la bibliothèque de communication la plus utilisée pour les applications parallèles, a été efficacement implémentée dans un contexte de clusters. Deux caractéristiques des grilles, les réseaux longue-distance et l'hétérogénéité des processeurs et des réseaux, posent la question de l'efficacité de MPI sur les grilles. Cet article présente une évaluation sur la grille de recherche française GRID'5000, de 4 implémentations récentes de MPI : MPICH2, MPICH-Madeleine, OpenMPI et GridMPI. La comparaison est basée sur un pingpong, les NAS Parallel Benchmarks. Nous mettons en évidence les différences de performance obtenues avec les 4 implé- mentations. GridMPI montre les meilleures performances. L'exécution d'applications MPI sur la grille peut être bénéfique à condition de régler finement certains paramètres des implémentations. Cet article détaille les para- mètres mis en jeu et leurs réglages

Exploring the Virtual Infrastructures as a Service concept with HIPerNET

With the expansion and convergence of communication and computing, dynamic provisioning of customized networking and processing infrastructures, as well as resource virtualization, are appealing concepts and technologies. Therefore, new models and tools are needed to allow users to create, trust and enjoy such on-demand virtual infrastructures within a wide area context. This research report presents the HIPerNET framework that we are designing and developing for creating, managing and controlling virtual infrastructures in the context of high-speed Internet. The key idea of this proposal is the combination of network- and system-virtualization associated with controlled resource reservation to provide fully isolated environments. HIPerNET's motivations and design principles are presented. We then examine specifically how this framework handles the virtual infrastructures, called Virtual Private eXecution Infrastructures (VPXI). To help specifying customized isolated infrastructures, HIPerNET relies on VXDL, a language for VPXI description and modeling which considers end-host resource as well as the virtual network topology interconnecting them, including virtual routers. We exemplify the VPXI specification, allocation and execution using a real large-scale distributed medical application. Experimental results obtained within the Grid'5000 testbed are presented and analyzed

Comparison and tuning of MPI implementations in a grid context

Today, clusters are often interconnected by long distance networks within grids to offer a huge number of available ressources to a range of users. MPI, the standard communication library used to write parallel applications, has been implemented for clusters. Two main features of grids: long distance networks and technological heterogeneity, raise the question of MPI efficiency in grids. This report presents an evaluation of four recent MPI implementations (MPICH2, MPICH-Madeleine, OpenMPI and GridMPI) in the french research grid: Grid'5000. The comparison is based on the execution of pingpong, NAS Parallel Benchmarks and a real application in geophysics. We show that this implementations present performance differences. Executing MPI applications on the grid can be beneficial if the parameters are well tuned. The paper details the tuning required on each implementation to get the best performances

Joint Elastic Cloud and Virtual Network Framework for Application Performance-cost Optimization

International audienceCloud computing infrastructures are providing resources on demand for tackling the needs of large-scale distributed applications. To adapt to the diversity of cloud infras- tructures and usage, new operation tools and models are needed. Estimating the amount of resources consumed by each application in particular is a difficult problem, both for end users who aim at minimizing their costs and infrastructure providers who aim at control- ling their resources allocation. Furthermore, network provision is generally not controlled on clouds. This paper describes a framework automating cloud resources allocation, deploy- ment and application execution control. It is based on a cost estimation model taking into account both virtual network and nodes managed by the cloud. The flexible provisioning of network resources permits the optimization of applications performance and infrastructure cost reduction. Four resource allocation strategies relying on the expertise that can be cap- tured in workflow-based applications are considered. Results of these strategies are confined virtual infrastructure descriptions that are interpreted by the HIPerNet engine responsible for allocating, reserving and configuring physical resources. The evaluation of this framework was carried out on the Aladdin/Grid'5000 testbed using a real application from the area of medical image analysis